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We propose a new adaptive quantization scheme for digitally 
implementing PCM and DPCM structures. The arithmetics we develop 
for the digital processing are useful as well in the implementation of 
previously existing schemes for adaptive quantization. Two objectives 
are stressed here: (i) The system must be robust in the presence of noise 
in the transmission channel which causes the synchronization between 
quantizer adaptations in the transmitter and receiver to deteriorate, 
(ii) It must also minimize the complexity of the digital realization. In 
addition to the above objectives, we require, of course, good fidelity of 
the processed speech waveform. The problem of synchronization in 
digital implementations where the constraint of finite precision 
arithmetic exists has not been addressed previously. We begin by ex- 
amining an existing, idealized adaptation algorithm which contains 
a leakage parameter for the purpose of deriving robustness. We prove 
that, to provide the necessary synchronization capability without 
impairing the quality of speech reproduction, it is necessary to use a 
minimum, unexpectedly large, number of bits in the machine words 
and, additionally, to carefully specify the internal arithmetic, as is done 
here. 

The new scheme that we propose here uses an order of magnitude 
less memory in an ROM-based implementation. The key innovations 
responsible for the improvement are: (0 modification of the adaptation 
algorithm to one where leakage is interleaved infrequently but at reg- 
ular intervals into the adaptation recursion; (ii) a specification of the 
internal machine arithmetic that guarantees synchronization in the 
presence of channel errors. A detailed theoretical analysis of the sta- 
tistical behavior of the proposed system for random inputs is given here. 
Results of a simulation of a realistic 16-level adaptive quantizer are 
reported. 



* A short version of this paper was presented at the International Conference on Com- 
munications, Toronto, June 1978. 
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I. INTRODUCTION 

We propose a new scheme for adaptive quantization which is partic- 
ularly well suited to the digital implementation of PCM and DPCM 
structures. In the course of this work, we have developed arithmetics for 
the digital processing that are useful as well in the implementation of 
previously existing schemes for robust quantization. 

The exacting requirements on adaptive quantization stemming from 
the broad dynamic range and rapid transient behavior of speech are well 
known. Two additional objectives are given equal importance here: (i) 
To make the system robust in the presence of channel errors. Thus, while 
channel errors may cause the quantizer adaptations in transmitter and 
receiver to be put out of synchronization,* a mechanism must exist which 
acts to rapidly restore the synchronization during periods of error-free 
transmission, (ii) To minimize the complexity of the digital realization; 
specifically, to minimize the length of the internal words in the digital 
processors and to facilitate the multiplexing of the hardware. 

Systems do exist in the literature for robust quantization in the 
presence of noisy channels; one such system is described below in some 
detail. However, the problem of synchronizing the quantizer adaptations 
in the transmitter and receiver in digital implementations, where the 
constraint of finite precision arithmetic exists, has not been addressed 
previously. We prove that, to provide the necessary synchronization 
capability without impairing the quality of speech reproduction, it is 
necessary to use an unexpectedly large number of bits in the internal 
words of the digital processors at both sites and, additionally, to carefully 
specify the internal arithmetic (which we do). If the digital processing 
is implemented using ROMs, as is being proposed, the long internal word 
length is reflected in large memory requirements and therefore costly 
implementations as well as exposure to new errors in the processing. 

The scheme that we propose here uses an order-of-magnitude less 
memory in an ROM-based implementation in both the transmitter and 
receiver. This is for comparable performance with respect to loading 
characteristic, signal-to-noise ratio, and the synchronization capability. 
Another advantage not reflected in the above estimate is the fact that 
the essential costly digital component, the ROM, as distinct from other 
less costly components such as adders, is used only for a small fraction 
of the total operating time. Thus, further economies may be effected 
through multiplexing the ROM. The key innovations are: (i) the modi- 
fication of the adaptation algorithm which allows the internal word 
length of the digital processors to be reduced significantly; and (ii) a 
specification of the internal arithmetic that guarantees synchronization 
in the presence of channel errors. As mentioned previously, the arith- 
metic is also applicable in digital implementations of previously existing 
adaptation algorithms. 



* In our usfckge, synchronization is synonymous with tracking. 
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A byproduct of the work reported here is that it establishes a link 
between two hitherto unconnected areas, namely, finite-arithmetic 
digital signal processing and waveform quantization in the presence of 
a noisy channel. The problem of synchronizing two geographically sep- 
arated digital processors gives rise to quite novel requirements on the 
processing, and we expect that the problem will be a subject of further 
investigation in the future. 

The paper is organized as follows. In Section 1.1 we describe an existing 
quantizer adaptation scheme and the associated synchronization 
problem. Section II is devoted to the basic description of the new scheme. 
Section 2.1 introduces the key idea underlying the scheme. Section 2.2 
considers the digital implementation of the system, and Section 2.3 
considers the synchronization behavior of the resulting system. Section 
III is devoted to the probabilistic analysis of the behavior of the proposed 
algorithm. The basic notions of the bias functions, central log step sizes, 
and load curves are introduced, and the qualitative results proved in their 
connection are stated. In Section IV, some computational results are 
presented in the context of a realistic 16-level quantizer that has been 
proposed and investigated previously in connection with an industrial 
application. We try to illuminate the topics considered in Sections II and 
III through examples involving this particular quantizer. Four appen- 
dices to the paper present the detailed technical derivations. 

On account of the length of the paper, we considered it desirable to 
include a final section, Section V, which summarizes and puts into per- 
spective the key results obtained in the preceding sections. 

We should mention that the digital implementation of adaptive DPCM 
systems is under investigation within Bell Laboratories in connection 
with TASI-D, subband voice coding, and new channel banks. The work 
reported here is a research study and not a description of a developed 
design. 

1. 1 Background and description of the problem 

We begin by describing a system proposed in Ref. 1 which, unlike 
earlier systems upon which it is based, 2-5 possesses the capacity to re- 
cover from past channel errors during periods of error-free transmis- 
sion. 

1 . 1. 1 An existing Idealized scheme for robust quantization 

Let A(i) (see Fig. 1) denote the step size of a quantizer, with 2N levels, 
at the ith sampling instant; A(i) is adapted according to the rule 

Mi + 1) = Mi) p Md), i = 0,1,2, • • • (1) 

where 0, < < 1, is the leakage constant and M(i) is the multiplier at 
time i. M(i) is selected from a prespecified collection of multipliers 
\Mi,M% ' ■ • ,M'n\ according to the rule: 
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Fig. 1 — The quantizer. A natural coding scheme is displayed. The step size is time- 
varying and the parameters (£„) and (7j,J are prespecified and fixed. 

If &-lA(0 < \x(i)\ < £ r A(i), then M(i) = M r , (2) 

where x(i) is the input signal variable (speech or data) at time i and 
= £o>£i» * • * &n-i&n = °° are fixed, ordered parameters of the quan- 
tizer,* Fig. 1. The multipliers are also ordered, i.e., 

Mi < M 2 < • • • < M N . 

It is widely recognized 6 - 7 that (1) is not in a form convenient for im- 
plementation, even analog implementation. To utilize conventional 
multipliers, it is necessary to work with the log-transformed version of 
(1). 

Denote the log step size by d(i), where 

d(i) ± log Q A(0, (3) 

Q being a fixed number greater than 1, and the log multipliers by 

m(i) = log Q M(i), m r = \og Q M r , 1 < r < N. 
Also let 

Zr = log Q £ r , 1 < r < N. 
Thus, from (1) and (2), 

d(i + 1) = 0d(i) + m(i), i = 0,1, - • - 
where 



(4) 

(5) 

(6a) 



* When the parameters |£ r | and |»j r ) are spaced equal distances apart, the quantizer is 
usually referred to as a uniform quantizer and it is natural to call A the "step size." How- 
ever, for nonuniform quantizers, the term "step size" is less natural and other candidates 
are "scale" and "range." However, since there is no reason for confusion, we retain the 
familiar term "step size." 
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m(i) = m r ffiir-i + d(i) < log Q |x(i)| <h + d(i). (6b) 

The only information that is coded and transmitted at time i is that 
concerning the quantizer output which uniquely determines the selected 
log multiplier m(i). A natural coding scheme is exhibited in Fig. 1. The 
recursion in (6) is implemented at both the transmitter and receiver. We 
let m'(i) denote the log multiplier corresponding to the received code 
word at time i, and we employ the natural notation d'{i) to denote the 
log step size in the receiver. The reconstruction, R(i), at the receiver of 
the input signal variable is done according to the rule: 

If ro'(i) = m r then \R{i)\ = v r Q d ' {i) , (?) 

where v r , 1 < r < N, are also prespecified, fixed parameters of the 
quantizer, as shown in Fig. 1. The sign of the reconstructed value is ob- 
tained from the sign bit, usually the first and shown as such in Fig. 1, in 
the received code word. 

The synchronization capability of the system, i.e., the capability 
possessed by the solutions of the recursions, \d(-)\ and \d'(-)\, at the 
transmitter and receiver to approach each other during error-free 
transmission is entirely due to the presence of the leakage parameter 0. 
For if d(0) and d'(0) are two, possibly different, initial values of the 
log-step sizes at the commencement of an epoch of error-free transmis- 
sion, then during the epoch 

\d(i) - d'(i)\ = 0»"|d(O) - d'(0)|, i > 0. (8) 

The notion of introducing leakage as a mechanism for deriving robust- 
ness in the presence of a noisy channel is a well-known one in commu- 
nication practice; witness, the leaky delta-modulator. 8 

As far as the synchronization of the transmitter and receiver adap- 
tations is concerned, eq. (8) implies that decreasing provides improved 
quality. However, there is an accompanying price. The data in Fig. 5 of 
Ref. 1 together with the theory developed here in Sections 3.2 and 3.3 
on the load curves (which describe the statistical behavior of the step 
size for random inputs) show that the statistical dynamic range of the 
step size is reduced rapidly with decreasing 0, with a concomitant de- 
terioration of the quality of the reconstruction.* Recent subjective tests 10 
have shown that it is very unlikely that /3 less than 6 %4 can provide ac- 
ceptable quality speech reproduction. 

Herein lies the gist of the problem: For good quality reproduction, the 
leakage parameter must necessarily be very close to 1, and this, on the 
other hand, makes it difficult to provide good quality synchronization. 
It is thus necessary to walk a narrow path between too small leakage and 
too large leakage. As we see next, the constraint of finite precision 



Numerous related topics are treated analytically in Ref. 9. 
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arithmetic imposed by a digital implementation compounds the design 
problem. 

1. 1.2 Digital Implementations 

Equation (6) assumes continuous values of d(-) and infinite precision 
arithmetical operations, and hence it can only serve as an ideal in a digital 
implementation. An all-digital coder will have only a limited dictionary 
or total number (typically, >32, <128) of possible log step sizes. We will 
consider the log step sizes to be integers varying from to 2 K — 1; thus, 
typically, 5 < K < 7. It is necessary to introduce the notion of an internal 
machine word with K integer bits and, say, F fractional bits (the need 
for fractional bits will become apparent shortly); the log step size is ob- 
tained from the internal machine word at time i, y(i), by means of an 
external arithmetic, such as truncation. Although later we will consider 
other possibilities, for the purpose of this discussion let us assume that 
the external word at time i, which is the log-step size at that time, is 
simply the integer part of the internal word at time i, i.e., 

d(i) = [ytOltruncate, i = 0,1,2 • • • . (9) 

The machine implementation of the ideal recursion in (6) is 

y(i + 1) = <0y(O> + m(i), i = 0,1,2, • • • , (10) 

where </3y (i) ) denotes some procedure, such as rounding, for taking fty (i) 
into a (K + F)-bit word. It will turn out later that this operation is best 
viewed with greater generality as a mapping / of (K + F)-bit words, with 
F fractional bits into other such words. Thus we restate (10) as* 

yd + D = Mi)] + »(0, i = 0,1,2, .... (100 

It will be assumed that all the log multipliers \m r \ have at most F frac- 
tional bits each, which ensures that if y(i) is a {K + F)-bit word then so 
isy(t + l). 

Figure 2 shows an example of the most direct procedure for generating 
the discrete map f(y), namely, by rounding #y to the nearest machine 
word. In the example, considered F = 1 so that the spacing between 
machine words is 2~ F = x /2- A feature common to such maps is that seg- 
ments of unit slope are juxtaposed between other segments of zero slope 
which we call "breaks." 

If, as before, we distinguish the quantities associated with the receiver 
by the superscript ', we see that the offset in the machine words behaves 



* In (10) and (10') we have not made allowances for overflow. This however can be done 
conventionally by employing saturation where: 

y(t+l) = Oif<0y(O>+m(O<O, 

= 2 K - 2~ F if (py(i)) + m(i) > 2 K - 2~ F , 
and in every other case (10) holds. Saturation acts to attenuate the offset in the machine 
words at the two sites. 
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as follows during epochs of error-free transmission [i.e., periods in which 

»(•) = »'(•)]: 

|y(i + 1) - y'(i + 1)| = |/|y(i)| - /{/(Oil (11) 

[compare with (8)]. 

The synchronization problem motivates us to impose the following 
two rather stringent requirements on the behavior of the offset. 
Synchronization requirements: 

(i) The offset is nonincreasing at all instants of error-free trans- 
mission. 
{ii) The integer parts of the machine words at the two sites, and hence 
the respective log step sizes, differ in at most a finite (preferably 
small) number of time instants during error-free transmission. 
We require the above to hold independent of the statistics of the input 
process. It is clear from (11) that these requirements imply restrictions 
on the discrete map / which are investigated below. 

Let us digress to better motivate the second of the above requirements. 
If the integer parts of the machine words at the two sites at any instant 
are not identical, then the respective log step sizes differ by at least unity 
and, hence, the ratio of the two step sizes is at least Q [see eq. (3)]; this 
factor may be unacceptably large since values of Q as high as 1.5 are being 
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Fig. 2 — An example of a naive machine arithmetic. 
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considered in practical designs.* To illustrate another facet of the second 
requirement, consider the case where, at a particular instant, the 
transmitter and receiver machine words are rather close, say, 1.9375 and 
2.0625 (F = 4). Yet the integer parts are 1 and 2, respectively. Thus the 
step sizes are Q and Q 2 , rather far apart. This example serves to illustrate 
that the mere proximity of the two machine words is not enough to 
guarantee that the log step sizes are identical. 

In the following discussion, we will need to know the value of L, an 
integer, which is such that 

l-2- L+1 </?< 1-2~ L ; 

if = %, as in Fig. 2, then L = 3 and if = 63 / 64 then L = 6. To simplify 
the following discussion, we shall assume that 

P = 1 - 2~ L , (12) 

i.e. /3e {%, 3 / 4 , 7 /s. ■ • '}> with this form for /3, L is the minimum number of 
fractional bits required for the binary representation of 0. The as- 
sumption on the form of /3 is unessential, and later in Section 2.2 we in- 
dicate that no difficulties are presented if /? is not of the assumed 
form. 

We give two different but connected reasons which separately lead 
to the rather consequential conclusion that F > L if the resulting system 
is to have certain essential properties, including the synchronization 
capability. The first reason stems directly from the synchronization 
requirements. We show that the latter requires the map / to incorporate 
certain contraction properties which in turn can be possible only if the 
internal machine word has at least L fractional bits. The second related 
reason is that fewer than L fractional bits gives rise to rounding errors 
in each iteration of the recursion which makes it hard to predict the ef- 
fective value of the leakage parameter. Recall from Section 1.1.1 the 
stringent requirements on the leakage parameter. 

Below we amplify both the above arguments. This discussion will 
motivate a more exact treatment in Section 2.2, which will also provide 
answers to the questions raised here. 

Consider (10') in conjunction with the synchronization requirements 
(i) and (ii). For the first of the synchronization requirements to be sat- 
isfied, it is apparent that it is necessary and sufficient that 

|/(y)-/(y')|<|y-y'| (13) 

for all machine words y and y'. We refer to the above property of the map 
/ as the weak contraction everywhere property. The map / shown in Fig. 



* This is the case if K is 5 or 6. If K is larger, then it is possible to relax the second re- 
quirement by requiring that the offset in the integer parts of the machine words be reduced 
to a small number (instead of requiring them to be identical). Thus it is possible to trade 
a higher K for a lower F while keeping K + F fixed. In any case, only minor modifications 
to the framework that is developed here will allow such cases to be handled. 
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2 possesses this property by virtue of the fact that the slope of the graph 
of/ is everywhere either (at the breaks) or 1. 

For the second of the synchronization requirements to be satisfied, 
we claim that it is necessary and sufficient that the map / have the fol- 
lowing property: 

If y and y' are any machine words with different integer parts, then 

\f(y)-f(y')\ £6\y-y'\ for some 5 <1. (14) 

We call the above the strong contraction across integer boundaries 
property. Sufficiency is clear, since we have that during epochs where 
the machine words do not have identical integer parts and error-free 
transmission exists, 

\y(i)-y'(i)\ <5'|y(0)-/(0)|. (15) 

Conversely, if (14) is not true, then it is easy to construct examples where 
the integer parts of the two machine words are different at an unbounded 
number of time instants. Referring to Fig. 2 we see that the graph of/ 
does not possess the strong contraction property (14). To illustrate, 
suppose that initially the two machine words have different integer parts 
and that both words occur in the range [2.5,6]; we see from the figure that 
no mechanism exists to prevent the two words from indefinitely re- 
maining in this range and simultaneously having different integer 
parts. 

We will now argue that the above two contraction properties, together 
with any weak fidelity criterion relating f(y) to fty, implies that F > L. 
Observe that the strong contraction property, (14), requires a "break" 
(see "breaks" in Fig. 2) in the graph of f(y) just prior to'euery integral 
value of y. Reason: y = k - 2~ F and y = k,k integral, have different in- 
teger parts. Further, if the local slope of the graph of f(y ) is not zero, then 
by virtue of the weak contraction property it is either 1 or —1. Finally, 
if F fractional bits are used, then each unit interval of y is composed of 
2 F intervals of equal length corresponding to that many distinct machine 
words. These three considerations show that the 

2 F — 1 
average slope of the graph of /(•) < — — — = 1 - 2~ F . (16) 

But f{y) is supposed to approximate fty , = 1 - 2~ L . Thus, just about 
any weak fidelity criterion will give that the smallest value of F, which 
allows the map / to have the properties required of it, is L. 

Our second reason is closely related to the aforementioned fidelity 
criterion. Implicit in a choice of a leakage parameter with a large 
number of fractional bits, L, in its binary representation (e.g., 13 = 63 / 64 ) 
is the requirement that the absolute rounding error in each iteration of 
(10'), \f\y(0\ - Py(i)\, be not larger (at least not by much) than an error 
in the least significant bit of 0, i.e. 2~ L : 
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\f(y) — py\ < 2~ L , for all machine words y. (17) 

Otherwise, there is no a priori need to specify (3 to that degree of preci- 
sion. (Our experience with the idealized system, discussed previously, 
shows that it is indeed necessary to specify /3 to a high degree of preci- 
sion.) A little thought will convince the reader that for such a bound, (17), 
on the rounding error to be valid it is necessary that the internal machine 
word have at least L fractional bits. 

In Section 2.2 we show that it is possible to obtain maps / with the 
weak and strong contraction properties that satisfy the fidelity criterion 
with the minimum possible number of fractional bits, i.e., F = L. We 
show that, in fact, the maps obtained are unique. The results will show 
that, for our maps, the offset in machine words during error-free trans- 
mission decreases exponentially fast to a value less than unity, after 
which there may be at most (2 L — 1) occasions at which the integer parts 
differ. 

Let us now consider in broad terms what the preceding results imply 
in terms of the cost and complexity of the digital implementation of the 
scheme for adaptive quantization discussed in Section 1.1.1. Consider 
the fairly typical case where the total number of integral log step sizes 
is 64 and £ = 63 /e4, i.e. K = 6 and L = 6. We now know that the total word 
length should be at least 12 bits. Consider the implications on the asso- 
ciated ROM size. The table stored in the ROM will have 2 12 addresses, each 
address containing 12 bits, giving a total memory size in the transmitter 
and receiver of about 50K bits each! Moreover, with each additional bit 
in the internal word, the memory requirement more than doubles.* 

In the next section, we propose a new adaptation algorithm and specify 
the required arithmetic. The new algorithm requires significantly fewer 
fractional bits in the machine words while possessing the necessary 
synchronization capability. 

II. THE PROPOSED SYSTEM 

2. 1 Idealized description 

We propose the following interleaved-leakage algorithm (ILA) as the 
basis for the machine adaptation of the log step size. For fixed parame- 
ters / and 7, / > 2 and < y < 1 [see eq. (6)]: 

d(i + 1) = yd(i) + m(i) 

d(i + 2) = d(i + 1) + m(i + 1) ]i = 0,1,21, (18) 

d(i + I) = d(i + I-l) + m(i + I-l) 



* We have considered the possibility of exploiting the idea due to Croisier et al. (Ref. 11) 
and Peled and Liu (Ref. 12) wherein the ROM size may be reduced at the cost of increased 
processing time. The processing times available and the relative costs do not make this 
approach particularly promising at the present time. However, it is an approach worth 
keeping in mind. 
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Here 7 is the leakage constant, and leakage is introduced only once in 
every / iterations. Thus we refer to / as the interleaving interval. The 
m(-) terms are the log multipliers, m(-) c {mi, • • • ,m N \, and the selection 
rule is as in (6b). However, in general, the optimum values of the mul- 
tipliers may be different from the ones in the scheme described in Section 
1.1.1 (we refer to the latter scheme as the uniform-leakage algorithm, 
or sometimes only as ULA). 

We observe that for two geographically separated implementations, 
\d(-)\ and \d'(-)\, of the recursion in (18) subject to possibly different initial 
values, d(0) and d'(0), but identical \m(-)\ sequences, as is the case during 
error-free transmission, we have for the offset, 

\d(i) - d'(i)\ = (T 1/7 )'|d(0) - d'(0)|, i = 0,1,21, .... (19) 

Comparing (19) with the similar expression in (8) for the offset in ULA, 
we find that the capability for recovery from channel errors is comparable 
in the two schemes if 

yl/I = 0. (20) 

The above is a key relation. Table I tabulates typical values of and 
the corresponding choices of 7 and / which give comparable recovery 
capabilities. There are small, inconsequential errors in the table which 
has been obtained from the approximation 7 = [1 - (1 - P)] 1 ~ 1 - 1(1 
— /3) for small values of (1 — #). 

The important point about the table is that, for given 0, the fractional 
bits required for a binary representation of the equivalent value of 7 is 
reduced by an additional bit for every doubling of the interleaving in- 
terval, /, in ILA. This simple fact is at the heart of the system that is 
proposed. 



Table I — Leakage parameters (0,7) and interleaving Intervals (I) 

for comparable synchronization capabilities in the uniform and 

interleaved leakage algorithms* 









7 (ILA) 






(ULA) 


7 = 2 


7 = 4 


7 = 8 


7 = 16 


7 = 32 


127 /l28 

6 %4 
31 /32 


6 %4 
31 /32 
15 /l6 


31 /32 
,5 /l6 

Vs 


15 Ae 
% 


Vs 


% 















* We have stopped short of using 7 = V2 for two reasons. First, there may be no advantage 
in reducing 7 beyond % because two fractional bits may be required in any case on account 
of the specification of the log multipliers, m r . Second, the change in the step size may be 
too drastic, and this may be reflected in the subjective quality. However, it is a possibility 
worth keeping in mind. 
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A slight generalization of the proposed scheme would have the mul- 
tiplier set in the iteration where leakage 7 is inserted to be different from 
the common multiplier set in all other iterations. This generalization 
provides no gain when the midpoint of the input signal intensities (a of 
Section IV) is scaled to be unity, which is the case considered in the 
simulations reported in Section IV. Goodman 10 has suggested that, when 
ff ± 1, the log multipliers in the leaky iterations be m(-) + (1 — 7) logQ<r, 
where \m(-)\ are the log multipliers in the nonleaking iterations. 

2.2 The digital Implementation 

We now consider the digital implementation of the idealized recursion 
(18). 

Here we let L, an integer, be such that 1 — 2~ L+1 < 7 < 1 — 2~ L . We 
make the simplifying, and inessential, assumption that 7 = 1 — 2~ L ; in 
this case, the binary representation of 7 requires L fractional bits. (Later 
we indicate through an example that it is easy to make the modifications 
which allow other values of 7 to be used.) Assume K integer and L 
fractional bits for the internal machine words. Thus, following the dis- 
cussion on the synchronization requirements in Section 1.1.2, we are 
assuming that the fractional bits in the machine words are the minimum 
necessary for the system objectives to be satisfied. Finally, assume that 
the log multipliers \m r \ are specified to L fractional bits. 

The internal description of the machine is 

y(i + i)=f\y(i)\ + m(i) \ 

yti + ?} ■ y(* tJLti^L t}) I l = 0,/ ' 2I >'"> (21) 

y(i + I)=y(i + I-1) + m{i + 1-1)) 

where y(«), the internal machine word, is a (K + L)-bit word with L 
fractional bits. In (21), f maps (K + L)-bit words with L fractional bits 
into other such words. The mapping / may be implemented most easily 
using ROMs; the characterization of the map / that we give below is a 
recipe for the programming of the ROMs.* 

The integral log step size d(-) is obtained from the internal word y (•) 
by a rule determined by an external arithmetic. We consider two natural 
and simple external arithmetics, rounding and truncation. Thus, 

Rounding: d(-) - [y(-)]round (22a) 

Truncation: d(-) = [y(0] truncate- (22b) 

We mean that if, for integral k,k -0.5 <y <k + 0.5, then [y] rc .und = k; 
\fk<y <k+l then [y] tr uncate = k. 



* Observe that the specifications of the maps given here and in Appendix A apply as well 
to the uniform leakage algorithm described in Section 1.1, provided /3 replaces 7 and the 
appropriate value of the parameter L associated with the leakage parameter in ULA is 
substituted. 
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We consider first the truncating external arithmetic. Following the 
discussion in Section 1.1.2, we impose the following requirements on the 
map /. (It is understood that all arguments of the map have L fractional 
bits.) 

(0 V<ri,cr 2 , |/(ffi) -/(cr 2 )| < ki " 0»|: 

"weak contraction everywhere." (23) 

(ii) a 1 t[k,k + 1) 

c 2 e[k + l,k + 2) _ l/(ai)-/(<r 2 )l . g < 
k integral |o"i — (r 2 | 

"strong contraction across integer boundaries." (24) 

(Hi) Vcr, \f(a)-ya\ <2~ L : 

"fidelity of discrete map to continuous map." (25) 

Recall from Section 1.1.2 that the first two properties are equivalent to 
the synchronization requirements. We also know that these two condi- 
tions together with almost any weak fidelity criterion relating f(a) to ya 
implies that the number of fractional bits in the machine words is at least 
L. We find that we can construct maps / which satisfy in addition the 
fidelity criterion in (Hi) without incurring the penalty of using more than 
L fractional bits. Also, as discussed previously, the fidelity criterion in 
(Hi) is important in itself. 

In Appendix A we give the complete specification of a map for each 
value of L. In Fig. 3a, we show the graph of the map / for the example of 

7 = %» where L = 2. In Appendix A we also show that there is only one 
such map / for any given L which satisfies conditions (i) to (Hi), (23) to 
(25). Further, for this unique map the value of the contraction parameter 

8 in (24) is 27/(1 + 7). 

When the external arithmetic is the rounding arithmetic (22a), the 




LOCATION OF 
"BREAKS" 



Fig. 3 — Machine arithmetics incorporating contraction properties and fidelity criterion 
for (a) truncating and (b) rounding external arithmetics. 7 = % and L = 2 (see Section 
2.2). 
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resulting map / is somewhat different. Appendix A gives the complete 
specifications of the maps for all values of L; these maps are also unique. 
Figure 3b shows the graph of one such map. 

Recall that earlier we made the simplifying assumption that 7 = 1 — 
2~ L . In general, L is defined to be such that 1 — 2~ L+1 < 7 < 1 — 2~ L . 
Figure 4 illustrates a map / for the case of 7 = % (L = 2) and the trun- 
cating external arithmetic. It may be verified that all the requirements 
in (23) to (25) are satisfied. We may similarly generate maps satisfying 
the requirements for arbitrary rational values of 7. 

Note that the maps obtained are rather special and quite distinct from 
the usual maps encountered in digital signal processing. 

Another point to note is that while we have specified arithmetics which 
use the minimum number of fractional bits, F = L, additional fractional 
bits, if they are available, may be put to use by incorporating more than 
one break in the graph off(a) per unit interval of a. The net effect is to 
give superior synchronization capability. 

Finally, note that the implementation of (21) requires by way of 
hardware only the ROMs, for implementing the map /, and adders. 
However, the ROMs are used only once in every / iterations. This provides 
an ideal opportunity for multiplexing the ROMs between different 
channels and different frequency bands in subband coding 13 applica- 
tions. 
2.3 Synchronization in the digital implementation 

We give some bounds on the offset between transmitter and receiver 
during periods of error-free transmission. 

By y and y', two machine words, having different integer parts we 

mean in the following that [y] roun d ^ b'lround Or [yltruncate 5* fr'] truncate, 

depending on the external arithmetic chosen. Thus, depending upon 
whether the two machine words have identical or different integer parts, 
the corresponding log step sizes are identical or different, respec- 
tively. 




LOCATION OF 
"BREAKS" - 



Fig. 4 — Machine arithmetic for 7 = % (L = 2) for two fractional bits in machine word 
and truncating external arithmetic. The contraction requirements and fidelity criterion 
are satisfied. 
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Suppose the machine implementations of the recursions in (18) in the 
transmitter and receiver during error-free transmission are: i - 
0,7,27, • • • 

y(i + l)=f\y(i)\ + m(i) 

yd + /) =y(i +7 - 1) + m(i "+ 7-1) 

/(i+l)=/i/(0} + m(0 

^l ( i ±_ 2 J jl^'Jl + i I +_™ <?± 1 1 

y'(i + I) = y'(i + 7 - 1) + m(i + 1 - 1). (26) 
Observe that 
\y(i + 7) - y'(i + 7)| \y(i + 1) - y'(i + 1)| 

= \f[y(i)}-f\y'(i)}\. 

Now from (23) and (24), 

l/MOI-fly'tOII 

— |y(0 — y'(*)| if y(0 an d y'(i) have identical integer parts, (27) 

< 8\y(i) — y'(i)\ if y(i) and y'(i) have different integer parts. (28) 

By repeated application of (28) we see that, if |y(0) — y'(0)\ > 1, 
then 

|y(/) " y'(j)\ < 1 for all; > / log ||y(0) - y'(0)||/log(l/«). (29) 

Thus, once the offset is reduced to less than unity it subsequently re- 
mains thus. 

Now consider the case where |y(0) — y'(0)| < 1. Consider the time 
instants j which are integral multiples of I. There can be at most (2 L — 
1) such time instants at which the integer parts differ. This is because 
a reduction of 2~ L in the offset is guaranteed by (28) in every such time 
instant. However, at time instants which are not integral multiples of 
I, the convergence of the integer parts is not quite as strong and is a 
penalty (which we believe to be insignificant) of ILA. 

III. ANALYSIS: PROBABILISTIC ASPECTS 

In this section, we investigate the probabilistic behavior of the log step 
sizes, \d(-)\, when the input signal variables, {*(•)}, are random and 
channel errors are absent. Clearly such an analysis is called for if we are 
to be able to guarantee certain qualitative features of performance that 
are basic and necessary in adaptive PCM systems. 4 - 5 The key notions of 
the bias function, central log step sizes, and load curves are introduced 
and their qualitative behavior pinned down. 

For our purposes here, the defining equations for the log step sizes are 
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in (18); the selection rule for the multipliers are in (6b). The key as- 
sumption that is made throughout this section is that {x (•)} is a sequence 
of independent, identically distributed random variables with mean zero 
and standard deviation a. We sometimes refer to a as the signal intensity. 
In keeping with the characteristics of speech, we are interested in a in 
the range of o- max /o- min = 100, or even 400 (40 and 52 dB ranges, respec- 
tively). 

3. 1 The bias function 

Define the bias function B(-\ a) to be 

B(d\a) = E[d{i + I)\d(i) = d]-d, i = 0,7,2/, • • • . (30) 

A little thought will show that the right-hand side of (30) does not depend 
on i — a consequence of the iid assumption on the input signal variables. 
Different values of a will generally yield different bias functions, which 
explains the notation. In engineering parlance, B(d\a) measures, for 
initial log step size d, the mean drift of the log step size after one cycle 
of updating of the log step size. 

We are able to show for a wide range of values of a that the bias func- 
tions consistently have a distinctive form, depicted in Fig. 5, of consid- 
erable significance. In particular, we show that B(d \ a) is positive when 
d is sufficiently small, and negative when d is sufficiently large. Further, 
under a rather mild restriction, we can prove the consequential result 
that B{d | a) is monotonic, decreasing with increasing d. The above results 
in their precise forms are proven in Appendix B. The restriction that is 
mentioned above is interesting in itself and, roughly, it calls for a pro- 
pensity for the expected log step sizes after one iteration to be ordered 
in the same way as the initial log step sizes. This turns out to require, 
roughly, that (m^ — mi) be not too large. 

The importance of the above results is on account of the following 
corollary which we state in qualitative terms: 

If (rn/v — mi) is not too large, then there exists a unique root, or 

zero-crossing, of the bias function B(-\ a). 
Without the monotonicity of the bias function, the possibility exists of 



B(d la- 




Fig. 5 — Sketch of a bias function. 
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there being many roots with a consequent dilution of the importance that 
we attach to the root. 

Let c denote such a root for a fixed value of a, Fig. 5: 
Definition of c: 

B(c|<r) = 0. (31) 

We refer to c as the central log step size (for signal intensity a). For a 
different value of a and hence a different bias function, the root will 
generally be different, and to make this dependence quite clear we use 
the notation c(a). 

As the terminology implies, we expect the probability distribution of 
the log step size to have a concentration of mass around c(o) whenever 
the signal intensity is a. The reason for expecting this (see direction of 
drift indicated by arrows at bottom of Fig. 5) is that, whenever the log 
step size is not at c(o), the mean drift of the log step size is toward 
c(a). 

The above conclusion is amply borne out by computational results 
(see Section IV). We find, for instance, that the fit between c(a) and the 
mean log step size in steady state is extremely good for a rather broad 
range of values of a. 

In summary, the dual properties of the central log step size (namely, 
that it predicts so well the mean log step size and that it is so much more 
tractable and easily obtained) explain the emphasis that we place on the 
notion of the central log step size. 

3. 1. 1 Method for generating the bias function 

The following recursive formula which is developed in Appendix B 
is the most effective method we know for obtaining the bias function. 
First, it is necessary to define the following functionals: 

b r (r) ± 2 f ^ p(n)d(n), l<r<N, (32) 

J*r-lQ T 

where p(ji) is the common pdf of the input signal variables \x (•)}. (It is 
slightly simpler to make as we do the inconsequential assumption that 
p(«) is symmetrical about 0.) Then B(d \ <r) is obtained as the solution of 
the following functional recursion: 

B o {d\<r) = 0, Vd 

| brWlBk-iid + m r \ a) + m r \, l<k<I-l 
B k {d\a) = Y~ l n (33) 

- (1 - y)d + E KidWk-^yd + m r \a) + m r \, k = I. 

r=l 

Finally,B(d|(r) = B/(d|(7). 
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The above formula is used in the following manner: Assume that the 
function Bk-i(d | a) is known for all values of d. Use (33) to generate next 
the complete function Bk (d \ a). After / such iterations, the resulting 
function B/(d | a) is in fact B(d \ a). 

The reader is referred to eq. (50), Appendix B, for the probabilistic 
interpretations of the ancillary functions Bk (*| a). 

The above formula is used in the analysis presented in Appendix B 
to determine the previously mentioned qualitative properties of the bias 
function B(d \ a). 

Figure 6 is a plot of the bias function B(d | 1) for a 16-level quantizer 
and normally distributed input signal variables. The interleaving in- 
terval, /, is 16. Observe in the figure that the graph is for d in the range 
[—200,800]. Values of d outside this range are not of much interest, since 
the maximum range of the log step sizes in this example is [Imi/(1 — y), 
ImjvAl - 7)] - [-163,828]. 

3.2 Load curves 

The load curves provide information regarding the manner in which 
the log step sizes depend on the input signalintensity, o\ We use the term 
to describe a graph of logQ<r vs. d, where d is the mean log step size in 
steady state for signal intensity a. Naturally, the range of a should cover 
the range of values expected in the specific application. 

From our previous discussion on bias functions and their roots, the 



200 




-100 



Fig. 6 — The bias function for uniform 16-level quantizer and normally distributed input 
signal variables, a = 1. Interleaving internal, / = 16 and 7 = 0.777. The log multipliers are 
given in (39). 
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central log step sizes, we expect a plot of logger vs. c( a) to be a rather good 
fit to the load curves. 

The utility of the load curve derives from the fact that it may be vi- 
sually compared with a plot of the ideal log step size with respect to a. 
This information may be obtained from solving a variational problem 
as is done by Max, 14 who has also tabulated the solutions for the case of 
normally distributed input signal variables. In any case, the solutions 
to the variational problem for the optimum log step size d(a) have the 
following form 

<2(<r) = log Q <r + A (34) 

where 2) is a constant which depends on the fixed parameters of the 
quantizer and, importantly, on the common pdf of the input signal 
variables. 

Figure 7 is a plot of the load curve obtained for the 16-level quantiz- 
er. 
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Fig. 7 — Load curve (d), central log step size (c), and approximate log step size (c ap „) for 
uniform 16-level quantizer and Gaussian, zero-mean, input signal variables of variance 
a 2 . The log multipliers are given in (39) and Q = 1.1. Interleaving interval, / = 16 and 
leakage, y = 0.777. 
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3.3 The almost-linear dependence of the central log step sizes on signal 
intensity 

Even though a plot of logQtr vs c(a) may be expected to be a rather 

good approximation to, and certainly simpler to obtain than, the load 

curve (logQa vs d), it is an unfortunate fact that it is not a very simple 

matter to obtain c(a). However, our graphs of c{a) have consistently 

displayed a most remarkable trait, namely, the almost-linearity of c(a) 

with respect to a. Intrigued by this feature, we found in an earlier study 9 

that it could be explained if the following rather unusual approximation 

is effective: 

*y 

p(p)dp » oti log y + a 2 , (35) 

'o 

where a\ and a 2 are constants and p (•) is the common pdf of the input 
signal variables scaled to have unit variance. 

Certainly, the above cannot be a good approximation when either y 
is very small or y is very large. But, as we see in Appendix C, we need the 
above to be a good approximation only for a limited range of y; specifi- 
cally, the range of y is required to include the range encountered by 
£iQ d(-) at one end, and &v-iQ d() at the other end, where d(«) is the typical 
log step size. It turns out that in the important cases where p(-) is either 
Gaussian or Laplacian, the range of validity of (35) is adequate, at least 
for the analysis of quantizers with up to 16 levels (N = 8). Further details 
may be found in Ref. 9. For both these distributions, we have found (35) 
to be an effective approximation in the range % < y < 2. For the former 
distribution, we have found good fits to be obtained if 

a x = 0.44 and a 2 = 0.34. 

(Below, we find it more convenient to express the rhs of (35) as ai logQ 
y + cx 2 .) 

With (35) as the sole approximation, in Appendix C we go through the 
involved and tedious process of approximating the bias function and 
thence deriving its root. The final result, however, is the following re- 
markably informative formula (c app (<r) is the approximate central log 
step size for signal intensity <r): 

CappM = S logqo + D, (36) 



where 



and 



S = (37) 

(1 - 7)U ~ 2ati(;ra N - mOK" 1 

1 — {1 — 2ai(rriN — mi)Y 

N-l 

m N -2 Y, ("V+i - ™ r )(on£ r + a 2 ) 
D= ^— S. (38) 
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Let us remark on certain features of the formula. Observe that, on 
account of ot\ being small, 1 — 2a\{m^ — mi) > almost certainly; for 
example, <x\ = 0.018 when Q [see eq. (3)] is 1.1 and the input signal 
variables are Gaussian. Consequently, we observe, from the formula in 
(37) for the slope S, that S < 1. Now the ideal slope is 1 [see (34)]. Thus 
eq. (37) expresses the undesirable but expected fact, alluded to earlier 
in Section 1.1, that decreasing the leakage parameter 7 has the effect of 
driving the load curve away from the ideal, as sketched in Fig. 8. 

As a digression, note that when 7 = 1, the slope S is unity. This is, of 
course, known to be the case. 4 - 5 We may also compare the expression for 
S with a similar expression for ULA derived in Ref. 9 — the two expres- 
sions are practically identical when 7 = /3 7 [eq. (20)] and is close to 
unity. This important fact, also confirmed in simulations in the example 
of Section IV, shows that in terms of the loading we expect the behavior 
in ILA and ULA to be roughly equivalent. 

One of the uses that formulas (36) to (38) can be put to is in the opti- 
mum choice of the multipliers. The approach we take is that 7 and (m^ 
— mi) are determined a priori on the basis of requirements arising from 
the quality of synchronization and transient response, respectively. This 
then fixes the value of S, eq. (37). However, there is still considerable 
freedom in the choice of the quantities (m r+ i — m r ), l<r<N — l, and 
thereby in the choice of the value of D, eq. (38). This degree of freedom 
may be exploited to determine the point of intersection of the graph of 
Capp(ff) and the ideal graph, which are shown in Fig. 8. A sensible choice 
for the point of intersection is at the signal intensity, a, that is most likely 
to be encountered. Usually, 1 this is at the midpoint of the range of signal 
intensities expected to be encountered in the application. 

IV. COMPUTED RESULTS 

Throughout this section, the input signal variables \x{-)\ are inde- 
pendent, Gaussian, random variables with mean zero and standard de- 
viation a. The signal intensity a is varied about a central value of 1.0. 

The quantizer is a 16-level, uniform quantizer, i.e., N = 8, £ r = r, 1 < 
r < N — 1, and i\ r = r — %, 1 < r < N. Throughout, the log base for the 
step sizes and multipliers, Q, is 1.1. 

'IDEAL LOG STEP SIZE, (34) 




LOGqCT 



Fig. 8 — The behavior of the central log step size compared to the ideal. See eqs. (34) and 
(36). 
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For the uniform-leakage algorithm, ULA, we used as the leakage con- 
stant /? = 6 %4. The multipliers for ULA are approximately those used by 
Rosenthal et al. 15 after correction, in the manner suggested in Ref. 1, for 
the following specifications: In the notation of Ref. 1, <r ■ midpoint of 
signal intensities = 1.0, the ideal loading factor = ideal step size/signal 
intensity = 0.257. This procedure gave the following values for the log- 
multipliers for ULA, 

m(l) = ro(2) = m(3) = ro(4) = -2.25; m(5) = ro(6) = 2.50; 

m(l) = 7.25; m(8) = 11.50. (39) 

The multipliers used for the interleaved algorithm, ILA, were also 
selected to be those given above. We are aware of the advantages of fine 
tuning the multipliers and Q to take advantage of the special features 
of ILA, but decided on balance to keep the multipliers and Q unchanged. 
We found that, as it stands, the transient behavior for ILA is slightly 
superior to that of ULA; reducing Q in ILA equalizes the transient be- 
havior in the two schemes and yields s/n ratios slightly better than those 
reported here for ILA. 



4. 1 Computed load curve, central log step sizes, and their approximation 

We illustrate the above notions for the interleaved leakage algorithm 
for the case of the interleaving interval, / = 16. We set 7_= /3 7 = 0.777. 
Figure 7 plots three quantities with respect to logQc: (i) d, the steady- 
state, mean log step size. This was obtained from 10,000 iterations; (ii) 
c((r), the central log step size defined in (31); (Hi) c app ((r), the approxi- 
mate central log step size as given by (36) to (38). 

For the given specifications, 

CappM = 0.99 log Q <r - 13.20. 

To clarify Fig. 7, we have also tabulated in Table II the values of the 
above variables at seven values of a. 

Table II — Computed load curve, central log step sizes, and their 
approximation (/ = 16 and 7 = 0.777) 

a, signal intensity 0.05 0.10 0.3162 1.0 3.162 10.0 20.0 

d, steady state mean log step -42.40 -35.53 -24.14 -12.46 -0.84 10.81 17.88 

S1Z6 

c(ff), central log step size -44.35 -37.07 -25.00 -12.93 -0.85 11.24 18.51 
c a?P (ff), approximate central -44.63 -37.36 -25.28 -13.20 -1.12 10.96 18.23 
log step size ^___ 
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4.2 S/N ratios and load curve for ULA and ILA 

Table III compares signal-to-noise ratios for the two schemes for a 
variety of interleaving intervals. The signal energy is simply the energy 
of the variables \x(-)\. The noise is exactly the difference between the 
input signal variable and its reconstruction at the receiver, assuming 
error-free transmission. Thus, the reported s/n ratios reflect the effect 
of the step-size adaptation algorithms but do not measure synchroni- 
zation capabilities of the systems — the latter is measured separately in 
Section 4.3. 

Note the almost identical s/n ratio performance for the two algorithms, 
ULA and ILA. 

Tables IV and V compare the mean and standard deviations of the 
log step sizes. Again, note the uniformity of the results for the ULA and 
ILA; the loading characteristics of the two approaches are almost iden- 
tical. 

Table III — Signal-to-noise ratios (dB) 

ila; 7 = 2 ILA; 7 = 4 ILA; 7 = 8 ILA; 7 = 16 
ULA 7 = /3 2 = 7 = j3 4 = 7 = 8 = 7 = 16 = 

a /3 = 63 /s4 0.969 0.939 0.881 0.777 
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Table IV — Steady-state mean log step sizes 

ILA; 7 = 2 ILA; 7 = 4 ILA; 7 = 8 ILA; 7 = 16 

ULA 7 = /3 2 = 7 = 4 = 7 = £ 8 = 7 = /3 16 = 

/3 = 63 / 64 0.969 0.939 0.881 0.777 
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Table V — Standard deviation of log step size in steady state 

ILA; 7 = 2 ILA; 7 = 4 ILA; 7 = 8 ILA; 7 = 16 

ULA 7 = /3 2 = 7=0 4 = 7=/3 8 = 7 = 16 = 

a /3 = 6 %4 0.969 0.939 0.881 0.777 
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4.3 The steady-state mean offset in the transmitter and receiver log step 
sizes 

Here we present some computational results connected with the 
steady state, joint distribution of the transmitter and receiver log step 
sizes assuming, as we have done throughout Section IV, that the input 
signal variables are independent, normally distributed. 

The channel is assumed to be memoryless; further, the event that a 
transmitted "1" is received as a "0" and the event that a transmitted "0" 
is received as a "1" have the common probability p. Thus, p is the bit 
error probability. In the numerical results presented below, the following 
typical value for the bit error probability is assumed: p = 10 -4 . 

Two geographically separated implementations of the interleaved 
leakage algorithm, (18), are assumed to be occurring: i = 0,7,27, • • • 
d(i + 1) = yd(i) + m(i) 

d(i + I) = d(i + I-l) + m(i + I-l) 

d'(i + l) = yd'(i) + m'(i) 

d'(i + 2) = d'(i + l) + m'(i+l) 
(40) 

d'(i + I) = d'(i + I-l) + m'(i + I-l) 

The information regarding the log multipliers m(-) are assumed to be 
coded in the manner shown in Fig. 1 and transmitted through the 
channel described above. The log multipliers ra'(-) are the log multipliers 
corresponding to the received code word. 

By the "steady state mean offset in the transmitter and receiver log 
step sizes" we mean the quantity e where 

e = \imE\d(i)-d'(i)\ (41) 

j— »<» 

In Appendix D we show that e is given by the following expres- 



sion: 



.9 



e=~ L (m r - m s )T srPn (42) 

1 _ Tr,s=l 



Table VI — Steady state mean offset in transmitter and receiver 
log step sizes. Bit error probability in channel, p = 10 -4 
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-0.020 


-0.021 


-0.022 


3.162 


-0.018 


-0.018 


-0.018 


-0.018 


-0.020 


10.0 


-0.015 


-0.015 


-0.015 


-0.016 


-0.017 
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where T = \T sr \ is the channel transition matrix given below and p r is 
the steady state probability that the rth code word is transmitted (00- • -0 
is the first code word, 11- • -1 is the last, Nth, code word; the sign bit is 
ignored). 

The channel transition matrix T is defined thus: 

T sr - Pr [sth code word reed. |rth code word trans.]. 

In the special case where the codes are as shown in Fig. 1, the elements 
of the matrix are obtained in a simple manner from the Hamming dis- 
tance between the code words. Thus, if d(s,r) is the Hamming distance 
between the sth and rth code words, then 

T sr = pd('.r)Q - p)log 2 N-d(s,r) ) 1 < S ,r < N. (43) 

In the example under consideration where N = 8, T\\ = (1 — p) 3 , T i2 = 
p(l-p) 2 ,etc. 

The formula given in (42) for e, the mean offset in log step sizes, is 
extremely useful. To see this, recall that e is defined in (41) in terms of 
the joint behavior of the transmitter and receiver in steady state, yet (42) 
provides the means for calculating e provided only that the transmitter 
log step size distribution is known, since the quantities \p r ] are statistics 
of the latter distribution. Thus, the considerably harder task of evalu- 
ating the joint distribution of the log step sizes at the two different sites 
is circumvented. 

Table VI enumerates the computed steady-state mean offset in 
transmitter and receiver log step sizes for various signal intensities and 
designs; note the almost identical performance. 

V. SUMMARY 

We consider it important that digitally implemented adaptive 
quantization systems possess two properties which, regardless of the 
statistics of the input signal, ensure that synchronization in the step-size 
adaptations at the transmitter and receiver is restored during periods 
of error-free transmission: The offset in step sizes is monotonic and 
nonincreasing and the step sizes differ in at most a finite number of 
sampling time instants. A detailed examination of the uniform-leakage 
algorithm (ULA) shows that a necessary and sufficient condition for the 
synchronization requirements to be satisfied is that the internal machine 
arithmetic, given by the nonlinear map /, possesses certain contraction 
properties. It is further shown that these contraction properties may exist 
only if the number of fractional bits (F) in the internal machine word 
is at least L where the leakage parameter is such that 1 - 2 _L+1 < 
< 1 - 2" L . Thus, if = 1 - 2 -L then L is the number of fractional bits 
required for the binary representation of 0. We proceed to show that it 
is actually possible to obtain internal machine arithmetics which satisfy 
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all the requirements with the minimum possible number of fractional 
bits, i.e., F = L. The arithmetics that we obtain are moreover unique. 
With these arithmetics the offset in machine words during error-free 
transmission decreases exponentially fast to a value less than unity, after 
which there may be at most (2 L — 1) occasions in which the step sizes 
differ. 

We give a complete specification of the unique maps /. Thus, in the 
case where truncation is used to obtain the log step size from the internal 
machine word, the formula that generates / is: 

If a = k + j2 -L , where k and j are integral and < j < 2 L — 1, then 

f(a) = k(l-2-^+j2-^. 

Figure 3a is the graph of the map / for the example of L = 2. 

Even the minimum length of the machine words translate into large 
memory requirements in ROM-based implementations. Thus, in the 
fairly typical case where the total number of step sizes is 64 and the 
leakage parameter (3 = 6 % 4 , we find that the minimum word length is 12 
bits, which translates into a ROM size of about 50K bits. 

We propose a new adaptation algorithm which is considerably more 
efficient in terms of the memory used in the implementation. In this 
algorithm, ILA, leakage is interleaved infrequently but at regular inter- 
vals into the recursion for the step-size adaptation. Thus, this scheme 
has as parameters 7, the leakage parameter, and /, the interleaving in- 
terval. We find that, for comparable synchronization capabilities in ULA 
and ILA, the parameters are related thus: 

yW = 0. 

Thus for /3 close to unity, 7 ~ 1 — 1(1 — /?)• Table I shows that for given 
13 the fractional bits required for the binary representation of the 
equivalent value of 7 is reduced by an additional bit for every doubling 
of the interleaving interval. 

To illustrate, consider the example given above where /3 = 6 % 4 ; the 
new scheme provides the option of interleaving leakage once in 8 itera- 
tions (/ = 8) with a leakage parameter 7 « 7 /s» which has three fractional 
bits. Thus, for the same total number of step sizes, the total word length 
required is 9 bits, which translates into an ROM size of about 5K bits and 
an order-of-magnitude reduction in memory size. Furthermore, the es- 
sential costly element of the system, the ROM, is used only once in 8 it- 
erations, thus allowing for the additional multiplexing of the ROM. 

The internal machine arithmetic that is proposed for ILA is identical 
to that specified for ULA, except that the machine word in the former 
system is of shorter length. 

A detailed theoretical analysis of the statistical behavior of the step 
sizes for independent random inputs is undertaken. Perhaps the most 
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insightful result obtained is a simple formula giving the approximate 
dependence on the input signal intensity, a, of the central log step size, 
c(a), which is the particular log step size about which the distribution 
of log step sizes is concentrated. The formula depends on only two pa- 
rameters, «i and «2> of the input signal distribution; in the case of 
Gaussian input distributions, a x « 0.44 log Q and a 2 » 0.34. This simple 
formula is given in (36) to (38). 

The idealized adaptation algorithms were simulated for a represen- 
tative 16-level quantizer and independent, Gaussian inputs. In the 
simulations, the multipliers in ILA were selected to be identical to those 
used in ula, although in general we expect the optimal multipliers to 
be different for the two schemes. The results of the simulations show 
that the performances of the systems are almost identical. 

APPENDIX A 

Specification of the Machine Arithmetics 

We describe first the maps / corresponding to the truncating external 
arithmetic in (22b) which satisfy conditions (i) to (jjj) given in (23) to 
(25), Section 2.2. In the example shown in Fig. 3a, observe that the 
breaks, i.e., zero slope segments between pairs of points, occur just prior 
to the integral values of a. This is also the rule by which / is obtained for 
general values of L. 

The following formula generates / for general values of L: 

If a = k + j2~ L , k and j integral and < ; < 2 L - 1, (44) 

then/(cr) = fe(l-2- L )+;2- L . 

Condition (i), (23), is trivially verified. For condition (ii), (24), note 
that for all integral k 

f(k + l- 2~ L ) - f(k + 1) = 0. (45) 

Thus a strong contraction across integer boundaries exists and, in fact, 
for a\ and a<i with different integer parts 

N-WI <JL, (46) 

| 0"i — flr 2 | 1+7 

so that we may take 

5 = 2y/(1 + y)< 1. (47) 

For the final condition {Hi), we find that 

< f{a) -ya< 2" L (1 - 2" L ), (48) 

where the two inequalities become equalities at a = k and a = k — 2~ L , 
respectively, whenever k is integral. 

We can also show rather easily that the map / given by (44) is unique, 
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i.e., there does not exist any other map satisfying the requirements (i) 
to (Hi). Uniqueness follows from the following two reasons: (a) Condition 
(ii) requires that there be a break in the graph off between a = k — 2~ L 
and a = k, k integral, i.e., f(k — 2~ L ) = f(k). Reason: a = k — 2~ L and 
a = k have different integer parts. (b) In order to satisfy at once both the 
fidelity condition (Hi) and the weak contraction (i) there can be at most 
one break in the typical integer interval [k, k + 1]. 

We now describe the slightly different map / which is obtained for the 
rounding external arithmetic, (22a). For the requirements on /, the only 
difference is in condition (ii) which now reads as follows: 

m °Ak - %k + y 2 ] i/( gl )-/( g2 )| 

a 2 e(k + V 2 ,k + %] |<ri-<r 2 | ~ 

The graph off shown in Fig. 3b is obviously similar to the one displayed 
in Fig. 3a, the main difference being the locations of the breaks which 
are here positioned immediately following the midpoint of the integer 
intervals. 

We rapidly summarize the key features of /. The formula for gener- 
ating / for general L is: 
If <r - k - V 2 + J2~ L , k and ;' integral, 1 < ;* < 2 L , 

then f(c) = k(l- 2~ L ) - V 2 + J2-L. (440 

The weak contraction condition (i) is trivially satisfied as well as the 
strong contraction condition (ii'), (240, with the same value of 6 that was 
previously obtained: 

5 = 2 T /(1 + t)< 1. (470 

Finally, 

\f(a) - ya\ < 2~^~\ (480 

and hence condition (Hi) is also satisfied. It is noteworthy that in keeping 
with the familiar properties of rounding and truncating, the above error 
bound is generally smaller than the corresponding bound in (48) for the 
truncating external arithmetic. 

The arguments used previously for establishing uniqueness apply as 
well for the above construction. 

APPENDIX B 

On the Bias Function 

We give here the derivations of the results on the bias function that 
are stated in Section 3.1, accompanied by more detailed insights and 
interpretations. It is convenient to drop the adjunct a in fi(«| a), the bias 
function, with the understanding that here a is arbitrary, but fixed. 
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B. 1 Generating the bias function 

We derive (33), which is a functional recursion yielding the bias 
function, 

B(d) = E[d(i)\d(0) = d]-d. (49) 

Define the ancillary functions 

B k (d) = E[d(I)\d(I -k) = d]-d,0<k<I, (50) 

so that 

B(d) = Bi(d). 
Observe that 

E[d(I)\d(I - k) - d] - £ sPr[d(/) = s|d(J - k) - d] 

= L Pr[d(J - Jfe + 1) - t\d(I -k) = d] 

t 

X E[d(I)\d(I - k + 1) = t], (51) 

where the Markov property has been used to obtain (51). Now t can take 
only N possible values. In fact, from (18), we see that if k <I, then te\d 
+ m r \r = 1, • • • ,N], and if k = I then te\yd + m r \r = 1, • • • ,iV]. Further, 
the respective probabilities are easily given in terms of the functionals 
b r (y), 1 < r < N, defined in (32), of the common pdf of the input signal 
variables. Thus, 

M<*)-Pr[*r-lQ d *|*(-)|<*rQ d ] 

[Pr[d(i -k + l) = d + m r \d(I - k) = d], 

1 <k <I -1 

(52) 
Pr[d(/ - k + 1) = yd + m r \d(I - k) = d], 

k=I. 

Substituting in (51), we arrive at the relations 
E[d(I)\d(I - k) = d] 

/ N 

£ b r (d)E[d(I)\d{I - k + 1) = d + m r ], l<k<I-l 

Z b r (d)E[d(I)\d(I -k+l) = yd + m r ], k = I. (53) 

r=l 

Substituting in the expressions in (50) for the functions Bk (•), we obtain 
the recursive formula given in the main text: 

B (d) = Q, 
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BM) = 



' £ b r (d)\B k -i(d + m r ) + m r \, 1 < k < I- 1 (54) 

r=l 

- (1 " i)d + £ M^fB/e-lM + ™r) + ™r}, ^ = /, (55) 

r=l 



andB(d) = B/(d). 

B.2 The range of the bias function 

Note that, as d -*■ -», the values of all the probabilities 61(d), • • • , 
b N -i(d) approach 0, while 6 N (d) — 1. Similarly, as d — °°, the values 
of all the probabilities b 2 (d), • • • ,b N (d) approach 0, while 61(d) -* 1. 
Thus* from (54) we have that 

Asd-^-co.B^d) -*m N , and as d -* <», Bi(d) -* m\. (56) 

Iterating, we obtain that 

As d -► -00, B 7 _i(d) -* (J - l)m N , 

as d — oo, B/_i(d) — (/ - l)roi. (57) 
Finally, for the bias function we obtain from the above and (55) that 
d ->-«>, B(d) « -(1 - y)d + Imi > 

d _^ 00> « -(l - 7 )d + Im N < 0. (58) 

The above is the basis for the claim that at least one zero-crossing of the 
bias function is guaranteed from observing the values of the function 
at the two limits. 

B.3 The monotonlclty of the bias function 

We establish here sufficient conditions which imply the rather im- 
portant monotonicity property of the bias function. Equations (54) and 
(55) provide the working definition of the bias function. Observe from 
(54) that for 1 < k < I - 1, 

B' k (d) = E KidWk-^d + m r ) + m r \ + £ b r {d)B' k . x (d + m r ) 

r=l r=l 

= - E K(d)[Bk-i(d + m r+1 ) - B k -i(d + m r ) + m r+l - m r \ + ". 

(59) 
We have found it convenient to introduce 

F r (d) = t b s {d), l<r<N. (60) 

The reason for this is that F' r {d) is positive since 

F r (d) = 2 f SrQ p(n)dn. (61) 

At this point, it is worth noting from (59) that B' k - X < is not enough 
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to establish that B' k < 0; it is necessary in addition that B' k -i be not ex- 
cessively negative. This motivates the bounding of the derivative of B k -\ 
from both below and above. We therefore introduce the quantities 

a k < min B' k (y); max B' k (y) < fa, (62) 

y y 

where it is understood that we are only interested in y having values in 
the finite dynamic range of the log step size. Further, let 

8(d) A ZKidHnir+i-mr) (63) 

r=l 

and 

< 5 min < 8(d) < 5 max . (64) 

From (59) we obtain 

B' k (d) < -«(d)(a*_i + 1) + fa-! 
^ -5 m in(afe-i + 1) + fa-i, assuming a k -\ < — 1. 
Thus, we may take 

0k = -5 min («A-i + 1) + fa-i, (65) 

provided a>k-i ^ —1. In identical fashion, we also obtain 

ock = -5 max (j8fc_i + 1) + a k -i, (66) 

again assuming ak-i ^ — 1. 

Summarizing, we have at this stage a coupled pair of recursions for 
the upper and lower bounds on the derivatives of the functions Bk, 1 < 
k<I — l, provided a k -\ >-l,l<k<I-l. Finally, we also have from 
(55) that 

B'(d) = B\(d) < (1 - 7 ) - 5 min («/-i + 1) + Tfo-i. (67) 

We may now solve the linear recursions in (65) and (66) for (oik, fa) 
with the initial conditions a = fa = 0. The following solution is obtained: 
1 < k £ 1, 

a fc -|{(l + 5)* + (l-5)*}-i.^.{(l + 5)*-(l-5)A}-l. (68) 

fa = I \(1 + 5)* + (1 - 5)*} - I • -*- • {(1 + «)* - (1 - 5)*} - l. (69) 
* * d max 

We have denoted by 6 the geometric mean of 5 max and 5 min , i.e., 

« = (5min5n,ax) 1/2 . (70) 

The reader will recall that the recursions (65) and (66) were contingent 
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upon a k -i > -1. We find, upon examining the "solutions," that we can 
ensure its validity over the range 1 < k < I - 1 provided a/_i > -1, 

q + g)/-i + (i-8'-i 

5 ma x<5. (l + - )/ _ 1 _ (1 _^ )/ _ 1 . (71) 

The above is a key relation. The first observation on it is that the 
relation implies not only that a/_i > -1 but also that /3/_i < 0, which 
is of primary interest. This may be verified either directly from the ex- 
pression in (69) or, more conveniently, from the recursion in (65) for fa 
and the fact that O = 0. But, as an examination for the bound on B'(d) 
in (67) shows, these two conclusions, namely, a/_i > -1 and #/_i < 0, 
are sufficient to guarantee that B'(d) < 0. We have thus arrived at the 
main result of this section: 

If 5 max satisfies the inequality (71), then B'(d) < 0. (72) 

Some insight into the nature of the inequality (71) may be gained by 
considering the case of 5 « 1. In this case, the rhs of (71) reduces to 1/(7 
- 1). Further, we observe from (68) and (69) that a k ~ -k8 max and fa 
« -fcSmin- Thus, summarizing, we have that 

If 5 « 1 then ot k « -k5 max , fa « -k8 min , 1 < k < I - 1 

and (71) requires that 5 max < 1/(7 - 1). (73) 

Thus, we have demonstrated that the monotonicity of the bias function 
is implied if the quantity 5(d) defined in (63) is uniformly small. 

Let us now examine the probabilistic import of the condition in (71), 
namely, that 

8(d) = 2F' r (d)(m r+1 - m r ) 

be not large. First, recall from the definition of F r (d) in (61) that 

F' r (d) = 2(ln Q)(ltrQ d )p(l;rQ d ), 1 < r < N - 1. (74) 

Thus, 

8(d) = 2(ln Q) N £ (m r+1 - m r )(£ r Q d )p(Z r Q d ) 

r=l 

= 2 N £ In (M r+l /M r )(Z r Q d )p(H r Q d ). (75) 

r=l 

Requiring that 8(d) be not too large is tantamount to requiring that the 
ratios of the multipliers, M r +JM r , be not too large. To make this con- 
nection quite transparent, we see that 

5(d) < 2 In (Mjv/Mi) TmaxypCy)!. (76) 
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For p(-) Gaussian with variance <r 2 , observe that 

max yp(y) = p(cr) = 0.242, (77) 

y 

so that, in this case, (76) states that 

8(d) < 0.484 In (M N /M{). (78) 

The above is not a particularly good bound, relative to the expression 
in (75), but it does illuminate the manner in which 5 max depends on the 
ratios of the multipliers. 

Finally, in summary let us recall in purely qualitative terms the reasons 
for requiring that 8(d) = "Z,F' r (d)(m r+ i — m r ) be not large. This condition 
is tied in a natural way to the conditions that B' k (d) > — 1, 1 < k < I — 
1, which is at the core of the above analysis since it follows rather easily 
from these conditions that B k (d) < 0, also. The conditions "B' k (y) > —1" 
have an entirely natural, underlying probabilistic interpretation. It 
merely states that, for two starting log step sizes, d (0) = d and d(0) = 
d', where, say, the ordering is d < d', the respective expected log step sizes 
after k iterations should also be ordered in the same way. A little thought 
is enough to convince one that such a condition can only be guaranteed 
by requiring that 8(d) be not too large, since 8(d) itself measures the 
potential for initial orderings to be reversed in one iteration. 

APPENDIX C 

Approximate Formula for the Central Log Step Sizes 

The object here is to derive the following approximate formula for the 
dependence of the central log step size on the signal intensity, o: 

c app ((7) = SlogQ<r + D, (79) 

where S and D, given in (37) and (38), are obtained from the fixed pa- 
rameters of the system. The sole approximation that is made is in ap- 
proximating the distribution of the input signal variables in the following 
manner: 

P(m)c//i * «i logQy + ot 2 , (80) 

o 



s. 



where p(«) is the pdf of the input signal variables normalized to have unit 
variance. 

The procedure that is followed consists of first deriving the approxi- 
mation to the bias function, using the recursive formula in (33), and 
subsequently deriving the root of the approximate bias function. Observe 
that the recursive formula in (33) calls for the quantities b r (-), 1 < r < 
N. We find it essential to work with the partial sums 
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F r (d)= L b s (d), l<r<N-l 

s=l 

XZrQ d 
p(n)dn 

* 2«i log Q (£ r Q d /<x) + 2a 2 , from (80), 

= 2a- [ d - 2«x logQa + (2a 2 + 2ai£ r ), (81) 

where a 2 is the variance of the input signal variables. Note that 
F N (d) = 1. 

Examining (33), we find that we may also write it as follows [for no- 
tational simplicity, we drop the adjunct a in Bk (d | a)]: 
for 1 < k < I - 1 

B k (d) = B k - X (d + m N ) + m N - £ F r (d)\B k -i(d + m r+1 ) 

r=l 

— Bk-i(d + m r ) + m r+ i — m r \. (82) 
Now suppose that Bk-i(d) may be expressed in the form 

Bft-i(d) = (A-i - l)d + gk-i logQff + h k - u (83) 

where (/*-i, 5*-i, /i*-i) do not depend on either d or a. Certainly, Bo(c0 
may be expressed in this form since Bo(d) = 0. We now show that Bk (d) 
may also be expressed in the above manner. 

Upon substituting the above expression for Bk-i(d) and the expression 
in (81) for F r (d), in (82) we find that 

B k (d) = (f k - Dd + g k log Q <r + h k , (84) 

where 

fk = {1 - 2ai(m N - mi)\fk-i, 
gk ■ Bk-\ + 2«i(miv - mi)fk-i, 



hk = hk-i + 



N-l 



fk-i. (85) 



m N -2 Y, (m r+ i - m r )(ai£r + « 2 ) 

l 

Certainly, the newly defined quantities are independent of d and logQtr. 
Thus, the basis exists for an inductive construction. Further, the coupled 
recursions in (85) are trivial to solve for the initial conditions /o = 1, go 
= 0, h = 0; thus, we obtain (//-i, gi-i, /i/-i). 

As is apparent from (23), the final iteration in the recursion for gen- 
erating the bias function differs from all the others. In fact, 

fi = \y- 2ai(m N - mj)\fi-i 

gl = gl-i + 2ai(m N - mi)//-! 

h I = h I -i + lm N -2 £ (m r+ i - m r )(a^ r + a 2 ) fi-\. (86) 

The complete solution for the approximation to the bias function 
is: 
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B(d) = {f i - l)d + gi log Q (T + h h (87) 

where 

U = -(1 - 7 ){1 - 2a 1 (m N - m^- 1 + {1 - 2a l (m N - mi)?, (88) 

gi = 1 -{1 - 2«i(m N - mi)} 7 , 

n-\ _ I 

mjv-2 I (m r +i- m r )(aiZ r + a 2 )\{l-{l-2ai(m N -mi)\ 1 } 



N-l 
fflN — 2 

hi = 



2a\{m.N — m,\) 

Recall that the central log step size is the root of the bias function B(d). 
Thus, denoting by c app (c) the root of the function in (87), we obtain 

Capp(^) = 7^-T logQtr + -^- (89) 

1 - / / 1 - // 

= S\og Q <T + D, (90) 

where S and D, trivially identified by comparing the two expressions, 
are as given in the main text, (37) and (38). 

APPENDIX D 

Formula for the Steady State, Mean Offset in Transmitter and Receiver Log 
Step Sizes 

We derive the formula for e given in (42). First, it is necessary to define 
certain quantities in connection with (40), which describes the step-size 
adaptations at the two sites. 

e(-) = d(-) - d'(-), the offset at time •, (91) 

and u(') - m(-) - m'(-), the offset in the log multipliers at time •. From 
(40) we obtain 

e(i + 1) = ye(i) + u(i) 

e(i + 2) = e(i+l) + u(i+l) ____\ i = 0,/,2/, ■ ■ ■ . (92) 

e{i + I) = e(i - I - 1) + u(i + I - 1) 

Thus, 

e(i + I) = yeii) + \u(i) + u(i + 1) + • • • + u(i + I - 1)\. (93) 
Taking expectations of both sides of the equation, 

e(i + I) = ye(i) + \u{i) + u{i + 1) + • •• + u(i + 1- 1)\, (94) 

where the bar has been used to denote mean values. 

Consider u(i), the first term inside the parentheses. Observe that 
u(«)«{m r — m A \ 1 < r,s < N\. Also, 

_... n f rth code word transmitted and sth 1 

u(i)= E (m r -m s )Pr . 

r,s= i L code word received at time i . J 
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w , . _ fsth code word reed, rth code "I 

= £ (m r -m s )Pr 
rs =i L word trans. J 

w _ frth code word trans. "I N 

X Pr . - £ (m r - m s )T sr p r {i), (95) 

L at time i J r ,s=i 

where T sr is simply the (s,r)th element of the channel transition matrix, 
and p r (i), 1 ^ r < N, is simply obtained from the pdf of the transmitter 
log step size at time i. 

Expressions for u(i + 1), • • • ,u(i + 1 — 1) may similarly be derived. 
Thus, for i = 0,1,21,- • • 

u(i) + ---u(i + I-D= L (m r - m s )T sr [p r (i) 

r,s=l 

+ .-. + p r (i + /-l)}. (96) 

To proceed further, it is necessary to assume ergodicity, i.e., more 
specifically, convergence in the mean for the time-evolving distributions 
of the transmitter log step size. With this assumption, as i —*■ <*> 

*(*) — « (97) 

and 

Pr (i) + • • • + Pr (i + / - 1 ) — Ip r , 1 < r < N, (98) 

where e and p r have the interpretations mentioned in the main text. 
Substituting in (94) and (96) yields 

e = — — E (rn r - m,)T ar p r , (32) 

1 - 7 r,s=l 

which is what we set out to establish. 
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